misc: Sync upstream Velox main - #65

Merge summary

• Upstream range: 231d16aa63c7dc924085603b02f7284bfae58796..796c6214c89a78ef86df29a64d4351129f62cb86
• Original sync base: open PR head 594e4d15360891aa1a67d183be484edbad78c572
• Recovered unpublished range: none
• Upstream integration: merge commit 9a350e1295c39b6470fe8431358710cefca56df1
• Latest downstream dev: merged freshly fetched cf619d90eca1b83ad31f446b96482df096d24923 in merge commit f95611c3ee9d3556e2a600aaf1faef20d487aba9
• Final Velox commit: f95611c3ee9d3556e2a600aaf1faef20d487aba9
• .github: retained exactly from the recorded latest downstream/dev; upstream .github changes were ignored.

Notable changes

• Core execution adds table-write NOT NULL enforcement and optional Unnest output names so unused outputs can be skipped. These API changes required the linked Spark-Gluten consumer adaptation.
• Parquet gains writer row-group sizing, Velox-reader coverage for Arrow serialization, faster unaligned DeltaBp SIMD loads, and expanded writer/metadata tests.
• Nimble bounds parallel decode across the reader tree, rejects shared dictionary encoding in index projection, and collects string min/max statistics in one pass.
• RPC fixes close-time stats access, low-water reporting, and AIMD recovery units; streaming aggregation avoids retaining clustered input when no group spans batches.
• Spark SQL adds map_from_arrays; date-time intervals over 24 days and Iceberg timestamp defaults receive correctness fixes.
• The latest downstream dev merge defers cuDF cache H2D work until after the submission lock and expands prefetch coverage.
• No additional Velox compatibility commit was needed after the two merges. NVIDIA/spark-gluten#172 adapts Spark-Gluten's plan converter to the new writer and Unnest APIs.

Conflict resolutions

• No manual Git conflicts.
• .github was restored/verified against the recorded baseline after the upstream merge and against freshly fetched downstream/dev after the incremental dev merge, per downstream-ownership policy.

Validation

• Status: BLOCKED_ON_SPARK_GLUTEN_FIX — smoke and full Colt pass against the unmerged fix below; this Velox PR remains Draft until that fix lands in Spark-Gluten dev and validation is rerun from scratch against dev.
• Spark-Gluten dev: 9f732196834cb3d1f67488dd1d2f93823802dc7a
• Spark-Gluten fix: NVIDIA/spark-gluten#172, tested SHA df462d247c8f5193d578ee8f8242d1a8c909748e (Ready).
• Colt: 73f99f12c9f0fe067f48a667a68e0c52e4dbd832
• Verification config: ebb16dcd6fcb238b34d39cd00a2eb46a9525d5ec; generated manifest 9b45526e6ac686c3fb1a95fdb2f94f1ae48e90b7113de4dcfe80d58f5835e338; provenance at /home/liangcail/velox-upstream-sync-20260814/results/20260904T022259Z/verify-config/provenance.json (SHA-256 e28bcbb0f83e7c716050f5468e377e3ddc8878276d763ca0b78ba5f139a0732d).
• Initial dev failure: build-libs, command /opt/colt-provider/spark-gluten-build-libs.sh --expected-gluten-revision=9f732196834cb3d1f67488dd1d2f93823802dc7a --expected-velox-revision=f95611c3ee9d3556e2a600aaf1faef20d487aba9 --cuda-arch=89 --bundle-mode=full, exit code 2. InsertTableHandle required notNullColumns, and UnnestNode required optional output names. Evidence: /home/liangcail/velox-upstream-sync-20260814/results/20260904T014815Z/smoke/verify/workflow/logs/10-build-libs/spark-gluten-build-libs.log.
• Repair: Spark-Gluten df462d247c8f5193d578ee8f8242d1a8c909748e supplies an empty notNullColumns set to preserve existing behavior and uses vector<optional<string>> for Unnest names. git diff --check, both clean consumer builds, and all downstream tests passed.
• Smoke, TPC-H Query 1: PASS, 361 source/native cases recorded (235 passed, 126 suite-declared skips) and Query 1 passed; /home/liangcail/velox-upstream-sync-20260814/results/20260904T022259Z/smoke/verify.
• Full, TPC-H Query 1–22: PASS, 361 source/native cases recorded (235 passed, 126 suite-declared skips) and all 22 queries passed; /home/liangcail/velox-upstream-sync-20260814/results/20260904T022259Z/full/verify.
• Validation receipt: /home/liangcail/velox-upstream-sync-20260814/results/20260904T022259Z/sync-validation.json.
• CUDA/GPU/images: CUDA 12.9; requested native, detected SM89 on GPU 0 (NVIDIA L4); driver mode auto resolved to host; cuDF 5beaa5954688fcb12236ffb434e192ea2c77db30; sealed dependency carrier sha256:a1fe82c81f79c981cde3d41ef9b5821884e1d1a93541b5d203523bf42f21882d; smoke/full build execution images sha256:5d225d9b71e02e3521a3b9dfc7258bf2348fead3ffc290a3e34d68f2ec1c3d61 and sha256:267f0c651d49cf9798d8cca331958c2e9e1e46743a40f1a5fc17b957024cebcf; smoke/full runtime image sha256:1b42a229c8357223f5c3bdc06dceee2ddddf6c3d0e692db1b5a8b8b074ffaaa3.